Discussion of the Papers by Rissanen and by Wallace and Dowe
نویسنده
چکیده
It is 12 years since I opened the discussion at the Royal Statistical Society meeting at which Rissanen and Wallace and Freeman presented companion papers on topics not far removed from those under discussion now. In the interim our understanding of the relationship between statistical inference, coding theory and algorithmic complexity has developed somewhat, but I feel there are still gaps to bridge. Rather than comment in detail on the current papers, I propose to use this opportunity to attempt a review of the whole area of stochastic complexity, or MDL, from a personal perspective.
منابع مشابه
Discussion of the Papers by Rissanen, and by Wallace and Dowe
Commenting on the papers by Dr Rissanen and Professors Wallace and Dowe is a daunting sort of pleasure. Superficially, the papers are not closely related: Dr Rissanen has focused on the Shtarkov optimality criterion in a minimum description length (MDL) context giving some new implications from normalization (see his equation (1)) for model selection and hypothesis testing. By contrast, Wallace...
متن کاملComplexity and Information-theoretic Approaches to Biology
Notions of information and/or complexity have been applied to the analysis of a broad spectrum of biologically relevant problems, such as protein structure prediction, DNA motif discovery and compression, and neural spike trainsa. These notions have been studied by Kolmogorov (1965) and Chaitin (1966), with Solomono (1964), Wallace (1968) and Rissanen (1978) applying these to problems of statis...
متن کاملDiscussion on Kolmogorov Complexity and Statistical Analysis
The question why and how probability theory can be applied to the real-world phenomena has been discussed for several centuries. When the algorithmic information theory was created, it became possible to discuss these problems in a more specific way. In particular, Li and Vitányi [6], Rissanen [3], Wallace and Dowe [7] have discussed the connection between Kolmogorov (algorithmic) complexity an...
متن کاملDiscussion of Paper 'Minimum Message Length and Kolmogorov Complexity' by C. S. Wallace and D. L. Dowe
where Q(y) is a prior probability for hypothesis y, −log Q(y) the ideal Shannon code length for it, and −log P(x | y) the same for the data x given the hypothesis y. It is customary to call negative logarithms of probabilities ‘ideal’ code lengths, because real code lengths must be natural numbers. It is true that if in the conditional Kolmogorov complexity K (x | y) of a string x , given anoth...
متن کاملRejoinder
We would like to thank all the discussants for their contributions. Our paper attempts only a first step towards a satisfactory linkage between complexity theory and coding-based inference and these discussions will help to blaze the paths for future exploration. We also take this opportunity to remind the reader that, in our opinion, MDL and MML agree on many, many points. While we certainly d...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Comput. J.
دوره 42 شماره
صفحات -
تاریخ انتشار 1999